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ABSTRACT 

This legislative report presents the results of a 
pilot project conducted ir eight suburban California school districts 
as part of an effort to develop a testing program identifying 
kindergartners with potential reading problems. The project was also 
intended to evaluate the program's effectiveness and gauge the 
feasibility of statewide implementation in light of the criteria of 
effectiveness, statewide utility, and cost. Five major findings 
indicated that: (1) a small number of kindergartners with potential 
reading problems was identified by teacher judgment instrument and 
screening tests; (2) the program was partially effective in correctly 
identifying kindergarten children with potential reading problems; 
(3) no single indicator of potential reading problems met the 
criterion of 75 percent correct identifications, regardless of the 
posttest measure used; (4) among single indicators, the kindergarten 
teacher judoment indicator, with the standardized test of reading 
achievement used as the posttest measure, came closest to meeting the 
criterion of 75 percent correct identifications; and (5) when the 
cutoff point on a commercial screening instrument was raised, the 
percentage of correct identifications decreased. It was concluded 
that the program was not effective, not advisable for statewide 
testing, and prohibitively expensive. (RH) 
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Forewcrd 



over the past four years a number of educational leaders and 
practitioners in California and 20 states as well as Canada have 
graciously cooperated in the development of this pilot project 
and in carrying out the numerous data collection efforts 
involved. In addition to the continuing efforts of the project 
coordinators and staff in the eight original districts, the 
project has received assistance from members of the advisory 
group of field experts and from school principals and teachers in 
over 300 schools in which project children were enrolled xn the 
spring 1988. 

Project staff in the Program Evaluation and Research 
Division have included: Tara Ballard, Terra Lee Black, Tammy 
Cabral, Mark Fetler, Sherry Gonzalez, Amy Houston, Roxanne Moger, 
Maura Reagan, Maureen Rolfs, Margaret Scheffelin, Carol Smith, 
and Miriam Torres. 

Special appreciation is expressed to Professor James 
Catterall of the Graduate School of Education of the University 
of California, Los Angeles, for his independent review and 
critique of the draft of this report. 

Interest in helping children in learning to read has been 
sho*/n in a number of ways during this pilot project. School 
districts responded to Assemblyman Sill Bradley's invitation to 
participate as volunteers in the pilot project. The willingness 
of parents to permit their children to participate in the pilot 
project is evidence of their concerns for all children to succeed 
in learning to read. The willingness of school personnel to 
cot tinue to collect and submit data on their students is a 
tribute to their interest in the topic and their abiding concern 
for assisting children who have sever.-? reading problems. 

William Padia Carolyn Fowie 

Division Director Proiect Director 



Program Evaluation and Research Division 



EXECUTIVE SUMMARY 



This legislative report presents the results of a pilot project 
conducted by the Department of Education under Chapter 1376, 
Stats. 85, (Assembly Bill 972, Bradley, see Appendix A). 
Conducted from November, 1985, through December, 1988, the pilot 
project accomplished its three objectives. 

The first objective was to develop and implement a testing 
program at the kindergarten grade level to identify children with 
potential reading problems at an early period. The pilot 
project's testing program was developed with the advice of a 
group of field experts in the areas of reading and learning 
problems (see Appendix B) . The testing program was designed to 
use A multiple indicator procedure with two types of screening 
instruments: teacher judgment and commercially published 
screening tests. The teacher judgment instrument relied on the 
daily observations of the child's performance in the teacher's 
classroo-i over a sustained period of instruction. The 
commercially published screening test instrriments relied on the 
child's performance during a particular testing session. 

The Pilot project's kindergarten testing program was implemented 
in eight midsized suburban school districts throughout the state, 
chosen from among volunteers. Within each school district, 
schools were selected to represent the range of socio-economic 
factors as defined in the California Assessment Program. Within 
each school, two kindergarten classes were selected. Within each 
kindergarten class, the kindergarten teacher selected children 
for possible participation in the pilot project's kindergarten 
testing program. Parents gave their permission for their 
children to participate. In spring, 1986, the multiple indicator 
procedures of the pilot project's testing program were 
administered to over one thousand English-speaking children in 4 8 
kindergarten classes in 24 schools. The field costs associated 
with the implementation of the pilot project's testing program 
totalled at least «32,810, or about $28 per participating 
kindergarten pupil. The greater portion of the field costs to 
the pilot project was associated with the commercial screening 
instrument indicator of the kindergarten testing program. The 
teacher judgment indicator was relatively inexpensive. (See 
Chapter II, page 6.) 

The second objective was to examine the effectiveness of the 
pilot project's kindergarten testing program in correctly 
identifying children at an early period. The effectiveness study 
was designed with the advice of the group of field experts as a 
pre-test — no prescribed intervention — post-test study over a 
two-year period, from spring, 1986, to spring, 1988. The 
effectiveness study was not designed to examine the nature, 
extent, or effectiveness of testing programs or screening 
procedures in current use. The study was designed to avoid 
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labelling of children, self-fulfilling expectations, and 
intrusion into the children's educational programs. The 
children's scores on the multiple indicators of the kindergarten 
testing program were used as the pre-test portion of the design. 
The post-test portion of the design was composed of two measures 
of reading achievement: teacher judgment and a nationally used 
standardized reading achievement test. The teacher judgment 
instniment relied on the teacher's daily observations of the 
child's performance in the classroom over a sustained period of 
instruction. The standardized reading achievement test 
instrument relied on the child's performance during a particular 
testing session. 

Intensive efforts were made to keep in touch with the schools ±n 
which pilot project children were enrolled or to which the 
children transferred, so that as many as possible of the children 
who participated in the spring 1986 data collection effort would 
be available to participate in the spring 1988 data collection. 
Schools administered the two post-test measures of reading 
achievement in the spring of 1988. Over 900 children in more 
than 300 schools participated fully in the effectiveness study. 

For the pilot project's kindergarten testing program to meet the 
criterion of effectiveness adopted for the study, at least 75% of 
the children identified by both of the multiple indicators in 
kindergarten as having potential reading problems haa to 
demonstrate actual reading achievement deficits on both of the 
measures of reading achievement at the end of the two-year 
period. The criterion of 75% was adopted on the basis that 50% 
is a chance level; 75% is 50% greater than chance. 

Five major findings emerged from the analyses of the data. 
First, a small number of kindergarten children was identified by 
both of the indicators of the pilot project's kindergarten 
testing program. The small number is reflective of the 
relatively small number of children identified by their 
kindergarten teachers as having potential reading problems, as 
compared with the number of children identified by the commercial 
screening instruments. 

Second, the pilot project's kindergarten testing program was 
partially effective in having correctly identified children in 
kindergarten as having potential reading problems. The multiple 
indicators of the kindergarten testing program (kindergarten 
teacher judgment and commercial screening instrument) met the 
criterion of 75% correct identifications only when the children's 
scores on a standardized test of reading achievement was used as 
the only post-test measure, as opposed to both standardized test 
and teacher judgment. 

The multiple indicators of the kindergarten testing program did 
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not meet the criterion of 75% correct identifications when 
teacher judgment of the children* s reading achievement was used 
as the only post-test measure. 

Third, no single indicator of potential reading problems used in 
the kindergarten testing program met the criterion of 75% correct 
identifications^ regardless of the post'test measure used as 
evidence of actual reading achievement deficit, the teacher's 
judgment, or the children's scores on the standardized test of 
reading achievement. 

Fourth, of the single indicators of potential reading problems, 
the kindergarten teacher judgment indicator came closest to 
meeting the criterion of 75% correct identifications. The 
teacher judgment indicator approached the criterion only when the 
standardized test of reading achievement was used as the post- 
test measure. 

Fifth, raising the cutoff point on the commercial screening 
instrument indicator of potential reading problems lowered the 
percentage of correct identifications. In other words, widening 
the net in kindergarten resulted in a lower percentage of correct 
identifications by the pilot project's kindergarten testing 
program. (See Chapter III, page 11.) 

The third objective was to draw a conclusion on the feasibility 
of statewide implementation of the pilot project's kindergarten 
testing program at the end of the pilot project, based on three 
criteria: effectiveness, statewide utility, and cost. The 
department's conclusions are based on the results of the pilot 
project and are not intended to apply to testing programs and 
screening procedures in current use. 

The pilot project's kindergarten testing program was partially 
effective in correctly identifying children with potential 
reading problems at an early period. The commercially published 
screening tests were not designed for use with the rapidly 
changing demographic patterns of children enrolled in California 
public schools. The costs for statewide implementation of the 
kindergarten testing program are estimated at $10.85 million. In 
addition to the costs incurred in the pilot project, there would 
be costs at local and stata levels for practical implementation 
of the kindergarten testing program. 

Given the limited effectiveness of the pilot project's 
kindergarten testing program, the increasing number of limited- 
English-proficient kindergarten children who would be excluded 
from the testing program, and the sizeable annual costs, it is 
not feasible to implement the pilot project's kindergarten 
testing program statewide. 

In view of the relatively greater effectiveness of the 
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kindergarten teacher judgement indicator of potential readincr 

klndera^r?2n l""^ l^^^^^Z^ administering the ^ 

kindergarten teacher judgement procedure, the Department 
recommends further developmental work to increase the 

correctTr?dL?ffv^n^^ judgement as one means of 

c,? J ? identifying children with potential readino oroblems 

nf^h??r^°?"'^2^*^ "^^^^ ac^^^nt ?he wide diversity 

c^ltSraf d?Cerf??v'or^M?i ^""^^"^ ""^ ^^'^ linguistic and '^"^ 
(Se^Sa^tariV %ge 19' California's public schools. 
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Chapter I. 
Introduction to the Pilot Project 



Purpose of the Report. This legislative report presents the 
results of a pilot project conducted by the Department of 
Education as authorized by Chapter 1376, Statutes of 1985 
(Assembly Bill 972, Bradley, see Appendix A). The pilot project 
was designed to accomplish three objectives. First, to develop a 
testing program to be utilized at the kindergarten grade level to 
determine which pupils have a potential for developing reading 
problems. Second, to conduct a study to examine the 
effectiveness of the testing program. Third, to draw conclusions 
about the feasibility for implementing the testing program 
statewide at the end of the pilot project. The pilot project waf^ 
not designed to examine the nature, extent, or effectiveness of 
testing programo or screening procedures in current use. 

The Department developed the testing program and the 
effectiveness study in consultation with experts in the areas cf 
learning and reading difficulties. The group of field experts 
(see Appendix B) was selected to advise the Department on the 
scope and design of the kindergarten testing program. The 
question addressed in the effectiveness study conducted by the 
pilot project was "Did at least 75% of the children identified by 
the kindergarten testing program as having potential reading 
problems demonstrate actual reading achievement deficits at the 
end of a two-year period?" The Deparument's conclusions on 
feasibility were based on three criteria: effectiveness, 
statewide utility, and cost. 

This chapter contains a statement of the problem; the background 
of the problem; and the background and limitations of the pilot 
project. Subsequent chapters describe the multiple indicator 
procedure kindergarten testing program developed and ir.plemented 
by the pilot project in eight midsized suburban school districts; 
the study of the effectiveness of the pilot project's 
kindergarten testing program; and the Department's conclusions 
regarding the feasibility of implementing the testing program 
statewide at the end of the pilot project. 

Statement of the Problem. For many years there has been an 
interest in identifying children during their kindergarten year 
for possible referral for intensive assessment and possible 
diagnosis of dyslexia (severe reading difficulties) . A variety 
of testing programs and screening procedures have been developed 
and used in these identification efforts. Although the interest 
in kindergarten testing and screening hars been high, little is 
known about the effectiveness of testing programs and screening 
measures in identifying young children who will later demonstrate 
severe reading difficulties. 
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Background of the Problen, Preparing to learn to read is a pre- 
academic task for young children. Success in reading is 
impozrtant to parents, teachers, and the children themselves. 
According to the School Readiness Task Force (California State 
Department of Education, 1988) , research indicates that there are 
universal sequences of growth and change that occur in children 
in the first nine years of life. At the same time, each child is 
unique with an individual pattern of growth as well as 
personality, learning style, and family background. Enormous 
variance exists in the timing of individual development that is 
within the normal range. Developmental ly approprxate programs 
and assessment practices are flexible in their expectations about 
when and how children will actjuire certain competencies, one 
important competency is learning to read; another important 
competency is being able to demonstrate reading achievement. It 
is not surprising that teachers, parents, and school 
administrators are interested in identifying children who have 
potential reading problems. 

A variety of screening :neasures to assist in identifying children 
who may have potential learning problems in important academic 
areas such as reading have been developed over the years by 
teachers, researchers r and commercial publishers. These 
screening measures are sometimes used in testing programs 
operated by school districts. Often, multiple indicators of 
potential reading problems are used; in other words, more than 
one screening measure is used. Testing programs include a number 
of activities, for example, selection or development of measures; 
purchase or duplication of testing and scoring materials; 
selection and training of examiners (and interpreters) to 
administer the measures; obtaining parent permission ro test each 
child; administration of the tests to children and monitoring of 
examiner (and interpreter) administration to assure adherence tc 
accepted standards' completion of forms; capturing teacher 
judgment based on classroom observation; scoring of the 
children's test booklets; completion of parent judgment forms 
based on observati'*-! at home; interpretation of the children's 
scores; writing individual reports on the performance of each 
child; informing, toy rr.ail, telephone, or personal conference, 
each child's parents cf the results of the testing of their 
child; and using the results of the screening as an indicator rf 
"potential reading problem" in order to make decisions on thf2 
possible need for further assessment of the children. A key 
element in district decision-making is the setting of cut-off 
points for determining individual children's need for further 
observation ("another look") or intensive assessment. 

The terms "screen" and "screening" have been used in many ways to 
mean a direct, or indirect, measurement of child performance or 
other characteristics. There are two main types of screening 
measures; measures relying on teacher judgment, such as 
observation checklists and rating scales; and measures which 
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obtain a direct measure of child performance. Teacher judgment 
measures rely on the teacher's knowledge of the child's 
performance over a sustained period of time in the classroom and 
the child's response to instruction. Direct measures of a 
child's performance, such as in a test, rely on the child's 
performance on a particular day and testing session. In 
districts operating kindergartens, screening measures are 
sometimes administered during the beginning or near the end of 
the kindergarten year. 

To be completely (100%) effective, a screening measure must 
identify as having potential reading problems only those children 
who will later demonstrate actual reading problems. The 
determination of the score for labelling a child as "has a 
potential reading problem," sometimes called the cut-off point, 
is an important ingredient of a judgment of "effectiveness" of 
the screening measure (Glass, 1978; Meehl, 1956; Meehl and Rosen, 
1955) . The higher the cut-off point adopted for assigning a 
child to a "potential reading problem" group, the more children 
will be assigned the label of "has a potential reading problem." 
Similarly, the lower the cut-off point, the fewer children will 
be assigned the label of "has a potential reading problem." in 
practice, the information gained through the administration of a 
screening measure (or a set of multiple indicators) is used to 
help decide which children need another look: or further 
assessment. 

Educational professionals and parents have been concerned about 
the progress of all children in the early primary school years. 
One of those concerns has been about the possible negative 
effects of incorrectly identifying a child (Goodwin and 
Driscoll, 1980). Incorrect identification as possibly having 
"difficulties" results in unnecessary (and costly) referral and 
assessment procedures, as well as the potential for inaccurate 
labelling of a child. According to a New York educator, 
kindergarten screening tests are used almost universally in his 
state and set expectations for students before they enter school 
(or first grade) . Despite the fact that the younger the students 
are, the less reliable their test results, great credence is 
given to these tests (Kelly, 1988). A Massachusetts educator 
recently expressed her concerns about labelling kindergarten 
children as "at risk" on the basis of a screening test (Martin, 
1988) . 

Another type of incorrect identification is that of identifying a 
child as not having "difficulties" when they might exist. 
Therefore, parents and educators have encouraged the development 
of screening methods which would help them notice children with 
potential special needs so that educational efforts may be 
focused on the child's particular strengths and limitations. 
Screening by its nature, therefore, is the "first stage" in a 
process designed to identify children with potential difficulties 
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or talents (in the case of this pilot project, potential reading 
problems) . Experts recommend that screening results be con- 
sidered only as a "flag" and not, of itself, a final decision 
about a child. To guard against inaccurate identifications and 
\innecessary, time-consuming assessments, multiple indicators are 
often used to gather information eUoout a child from different 
perspectives, such as teacher judgment and standardized screening 
tests. Logically, a diagnosis of severe reading difficulties (or 
dyslexia) would be expected to lead to specialized instruction or 
other interventions designed to ameliorate the negative effects 
of the reading difficulties. The authors of a recent national 
report on learning disabilities called for larification of the 
terms used in the field, including dyslexia, and more research to 
improve the current state of the art in assessment ( Learning 
Disabilitieai A Report To The U.S. Congress. Prepared by the 
Interagency Committee on Learning Disabilities. Washington D.C: 
Department of Health and Hi;^an Services, 1987). 

A search of the educational research literature revealed few 
studies in which a large number of kindergarten children in a 
number of schools and school districts were followed over a 
multiple-year period to examine the relationships among reading 
achievement deficits and earlier identification of children by 
screening measures. Previous studies of the relationships 
between scores on screening measures and subsequent reading 
achievement deficits have typically been performed with a sample 
size of 300 or fewer pupils, in only one school or in just a few 
schools, in one or a few school districts, and over a period of 
only one school year (Lichtenatein and Ireton, 1984) . 

No studies were found on the cost-effectiveness of administering 
screening procedures, such as in reducing the need for later 
remedial education. No studies were found on the costs sx cost- 
effectiveness of individual assessment for those pupils 
identified by the screening procedure as "having a potential 
reading problem" nor the costs of providing needed supplemental 
instruction to those pupils diagnosed and determined to be in 
need. 

One study was located on the implementation of a statewide 
mandate for screening new entrants to schools, including 
kindergarten and first grade students ( Screenin g of Public School 
Children. 1985) . This New York study found substantial compliance 
with the mandate, difficulties in screening children who spoke a 
language other than English, a variety of local procedures for 
setting cutoff scores for identification of children for further 
assessment, a low rate (1%) of children referred for further 
assessment as a result of screening, and a wide range of costs 
reported by school districts. Local costs of purchasing or 
developing screening instruments, training of examiners and 
interpreters, administration and scoring of tests, and writing 
reports based on the test results were partially covered by New 
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York State *s appropriations of more than nine million dollars 
each year. The proportion of costs devoted to kindergarten 
screening could not be determined from local fiscal reports. 

Background and Limitat ions of the Pilot Project. The Department 
conducted the pilot project from November, 1985 through December, 
1988. The department's data collection efforts for the pilot 
project's kindergarten testing program and effectiveness study 
were supplemented by in-kind contributions of staff time and 
resources by participating school districts and schools. The 
Department's pilot project was not intended to describe or 
examine testing programs and screening procedures in current use. 

The multiple indicators of the pilot project's kindergarten 
testing program were teacher judgment and standardized testing of 
pupils. The effectiveness study was conducted using a pre-test 
— no prescribed intervention — post-test design. The pre-test 
data were the kindergarten teacher judgment ratings and the 
pupils' scores from the commercially published screening test, 
gleaned from the kindergarten testi^-g performed in the spring of 
1986. The post-test data were scores made by the pupils in the 
spring of 1988 on a standardized test of reading achievement and 
their teachers' ratings on a project-designed evaluation form. 

The legislation directed the Department to report on the results 
of the pilot project on or before January 1, 1989. The 
Department's repoirt was to include, but not be limited to, an 
assessment of the degree to which the early diagnosis of learning 
disabilities in pupils participating in the pilot project 
resulted in a reduction of the need for later remedial education 
for those pupils and the projected cost-effectiveness of this 
early diagnosis. Regarding the issue of diagnosis of learning 
disabilities, and the cost-effectiveness of early diagnoses, the 
resources available to the Department's pilot project were 
insufficient to conduct the intensive case studies of individual 
children, including the lengthy assessment and diagnostic 
procedures necessary to determine the presence or absence of 
learning disabilities. In addition, the Department determined 
that prior to assessing the effects of an early diagnosis of 
learning disabilities, based in part on the results of the 
testing program developed and implemented by the pilot project, 
it was important to determine the effectiveness of the testing 
program. 

The next chapter presents a description of the kindergarten 
testing program developed and implemented by the Department's 
pilot project. 



Chapter II. 
The Kindergarten Testing Program 



This chapter describes the kindergarten testing program developed 
and implemented by the pilot project. Separate sections describe 
the development and the implementation of the testing program, 
followed by a summary. 

Development of the Pilot Project's Kindergar ten Testing Program 

This section describes the design of the pilot project's 
kindergarten testing program and the screening instruments used 
to rate and test the participating kindergarten children. In 
accordance with the authorizing legislation, an advisory group of 
experts in the areas of learning and reading difficulties, 
including neurologists, psychologists, persons working in the 
these areas in post-secondary educational institutions, teachers, 
school nurses, education consultants, and school psychologists 
had been established. Following the advisory group's recom- 
mendations, the testing program was designed to include multiple 
indicators of potential reading problems; kindergarten teacher 
judgment on an evaluation form and kindergarten pupil performance 
on a commercially published screening test. Thus for each 
participating child there would be multiple indications of 
potential reading problems. To avoid labelling a child, no 
information was to be given by the pilot project to the schools 
or teachers about the child's performance (or score) on the 
commercially published screening t^st. 

Four existing screening instruments were selected for use in each 
kindergarten classroom participating in the pilot project: a 
subjective kindergarten teacher evaluation instrximent focusing cn 
a child's reading readiness and three commercial screening 
instruments focusing on a child's direct performance. Each child 
was to be rated by the kindergarten classroom teacher and be 
administered one of the three commercial instruments. The 
primary criteria for selection of the commercially published 
screenin'/ instruments were established reliability and validity. 
Secondary criteria were brevity, ease of administration, ease of 
scoring, and reasonable cost. The four instruments are briefly 
described in the paragraphs below. 

The kindergarten teacher evaluation instrument, developed in a 
research project at the University of California at Los Angeles, 
sought the impressions of the kindergarten teacher of the 
children assigned to his or her classroom and requested a rating 
of each child's functioning in reading readiness and language 
development (Keogh and Sbordone, 1974) . The form is one page, 
double-sided. The teacher's time required to list and rate ee.ch 
child is approximately five minutes. The rating was scaled from 
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1 to 5. A rating of 1 indicated that the child was functioning 
poorly. The rating of 2 meant that the child was slightly below 
average. The rating of 3 meant that the child was about 
average. The rating of 4 meant that the child was slightly 
above average, while a rating of 5 meant the child was doing very 
well. 

The Florida Kindergarten Screening Battery ( FLORIDA ^ is a 20 
minute individual ccreening device. The expanded kit contains 
the basic kit with 50 record forms plua the Peabodv Picture 
YQSat?Ulary T?st - Revised (ppVT-R) and the Beerv Visual -Motor 
Integration (BemL_VMI) • The FLORIDA yields a score ranging from 
-3.0 to -HS.O. As stated by the authors, the FLORIDA is designed 
to "... predict the likelihood that an individual kindergarten 
child will manifest learning problems three years later (end of 
grade 2)" (Satz & Fletcher, 1982, p. 1). 

A Scanning Instrument for the Identification of Potential Learn- 
ing Disab rlity ( SEARCH ) is a 20 minute individual screening 
device, 'xa. complete kit contains a manual, toys, and record 
form. The SEARCH yields a score with a ran^e from 1 to 9. The 
authors have stated that the screening results from the SEARCH 
may be useful to "... predict learning difficulties in individual 
children ..." (Silver & Hagin, 1981, p.2) . 

The Revised Pre-Reading Screening Procedures to Identify First 
Grade Academic Needs ( SLINGERLAND ) is a group or an individual 
screening instrument which takes from 60 to 80 minutes to 
administer. For group administration an assistant for the 
examiner is required. For either process of administration, each 
child's test booklet is scored individually. In this pilot 
project, the SLINGERLAND was administered in a group setting in 2 
or 3 testing periods. The complete set of materials include test 
record forms, a set of cards, directions for administration, 
technical manual, and teacher's manual. The SLINGERLAND yields a 
score ranging from 0 to 130. As stated by the author the 
SLINGERLAND screening instrument is for the purpose of 

identifying those whose individual performances indicate modality 
weaknesses that call for specific instruction to prevent early 
failure." (Slingerland, 1977, p.l). 

Implementation of the Pilot Project's Testing Program 

This section describes the procedures used by the pilot project 
to implement the kindergarten testing program. The first 
subsection describes the procedures used to select the districts, 
schools, kindergarten classes, and children for participation in 
the implementation of the kindergarten testing program. The 
second subsection describes the procedures used by the pilot 
project and the eight cooperating school districts to administer 
the screening instruments to the participating children. The 
third subsection describes the costs to the pilot project of 

7 



ERIC 



implementing the kindergarten testing program in 48 kindergarten 
classes. 

Selection of Participants in th e Pilot Project. In November 
1985, all 916 California school districts maintaining 
kindergartens were invited by Assemblyman Bill Bradley to apply 
to the Department for participation in the pile: project of the 
kindergarten testing program- Of the 56 school districts that 
volunteered to participate in the pilot project, eight scliool 
districts were selected in February 1986. All eight were mid- 
sized suburban school districts; their enrollments ranged from 
9,000 to 20,000 students. Each district operated at least three 
schools having at least two kindergarten classes. The school 
districts also represented a range of geographical characteris- 
tics from northern, central, and southern California: Alum Rock 
Union Elementary, Bakersfield City Elementary, Burbank unified, 
Cupertino Union Elementary, Escondido Union, Fullerton 
Elementary, La Mesa-Spring Valley, and Ocean View Elementary. 

Each cooperating school district assigned an existing staff 
member to act as the pilot project coordinator. In collaboration 
with each district coordinator, three elementary schools were 
selected to represent a range of socio-economic factors. Within 
each school, two kindergarten classes were selected at random. 
The 24 schools selected equally represented low, medium, and high 
socio-economic areas, as defined by the California Assessment 
Program (CAP) . The teachers of each of the 48 kindergarten 
classes selected children to participate in the pilot project. 
The teachers were instructed to complete a project designed class 
list, excluding children the teacher thought not appropriate tor 
the project, such as having suspected mental retardation, 
debilitating physical handicap, or inability to speak or 
understand English. Parents gave permission for their children 
to participate in the pilot project. 

procedures used to Administer the Screening instruments t o 
Participating Children. In the spring of 1986, near the end of 
their kindergarten year, the participating kindergarten children 
were rated by their teachers and tested by specially hired and 
trained examiners. The 48 kindergarten teachers rated their 
children on the kindergarten teacher evaluation form. The school 
district coordinators forwarded the completed rating forms to the 
pilot project. 

Ei^ht examiners were hired by the pilot project to administer the 
commercial instruments. The examiners were recommended by 
members of the group of field experts; were graduate students; 
were knowledgeable about testing? and had experience in working 
with young children. Each of the examiners was trained to 
administer all three instruments in a two-d^y training session 
conducted by experts in each of the cimmercial instr\iments . 
After being trained, the examiners adn inistered the three 
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cosmercial screening instniments to 1^187 kindergarten children. 
Within each of the 24 classrooms, children were randomly as>}igned 
to one of the three coiamercial screens. The examiners forwarded 
the completed test booklets and scoring sheets to the pilot 
project. No information on the children's performance was 
provided to the participating districts, schools, or teachers. 

At the end of the spring, 1986 implementation period, the pilot 
project had received a completed kindergarten teacher evaluation 
form and a completed commercial screening test booklet and 
scoring sheet for 1,180 out of the 1,187 children. These 1,180 
children had participated fully in the kindergarten testing 
progrzua and were thus classified as participants in the 
effectiveness study conducted by the pilot project, to be 
described in the next chapter. 

Costs for Implementing the Pilot Project's Multiple Indicator 
Testing Program. The field activities of the implementation of 
the testing program had associated costs of at least $32,810. 
These activities and their costs were: 

a. orientation of kindergarten teachers to the pilot project and 
the testing program and time to complete the kindergarten teacher 
evaluation form — $960 for time for 48 teachers for orientation 
to the testing program [the time needed for the teacher^i to rate 
their children is estimated at 98 person-hours (1,180 children at 
five minutes each) ] ; 

b. purchase of commercial testing materials (Si, 980, including, 
for each of the three commercial tests, a kit for each of the 
eight examiners and test booklets for each of the 1,187 pupils 
who were administered one of the tests [these costs were offset 
by contributions by the test publishers]); 

c. training the examiners ($2,025 for a three-day training 
session conducted by three experts, one for each test) ; 

d. examiner participation in training and in administration and 
scoring of the tests — $21,440 for 320 days of examiner rime 
[the time needed for the examiners to remove the children from 
the kindergarten classroom, administer the tests, return the 
children to their classrooms, and score the test booklets is 
estimated at 1,187 hours (1,187 children at one hour each)]; 

e. travel for examiners and travel by the monitor of the 
administration ($6,400 for nine persons); and 

f. duplication of the teacher judgment form ($5.00 to duplicate 
the form for the 48 kindergarten teachers.) 

Because the pilot project was not designed to provide infonnation 
on the scores made by the children on the commercially published 



screening test indicator, there were no costs for interpreting 
the test results of each child; writing individual reports on 
each child; or reporting the interpretations to the parents of 
each child. 

Districts contributed the time of existing staff: coordinator, 
principal, Kindergarten teacher, and instructional aide time, as 
well as local costs for supporting pilot project participation. 
For example, the parents of each child in the 48 kindergarten 
classes were mailed a form explaining the pilot project and 
requesting parental permission for the child to participate. No 
cost data were collected from the cooperating district:;. 



Summary 

The pilot project's kindergarten testing program was implemented 
in spring, 1986 in 48 kindergarten classes in 24 schools in eight 
districts. The screening instruments used as the multiple 
indicators of the kindergarten testing program were administered 
to 1,180 kindergarten children at a cost to the pilot project of 
$32,81C or approximately $28 per pupil. The greater portion of 
the field costs to the pilot project was associated with the 
commercial screening instrument indicator of the kindergarten 
testing program. The teacher judgment indicator was relatively- 
inexpensive. 
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Chapter III. 
The Effectiveness study 



The purpose of this chapter is to summarize the study of the 
effectiveness of the kindergarten testing program developed and 
implemented by the pilot project. The effectiveness study was 
cSnduc?ed by the pilot prSject from spring 1986 through spring 
1988, The purpose of the study was to determine the 
effectiveness of the pilot project's kindergarten testing program 
in correctly identifying children at an early Period. The 
question addressed in the effectiveness study was "Did at least 
75% of the children identified by the kindergarten testing 
program as having potential reading problems demonstrate actual 
reading achi.3vem2nt deceits at the end of % two-year period?" 
In separate sections this chapter describes the methods and 
findings of the effectiveness study, followed by a discussion and 



summary . 



Methods 



'^he effectiveness study was designed with the advice of the group 
of field experts as a pre-test — no prescribed intervention — 
post-test study over a two-year period, from spring/1986 to 
spring, 1988. The effectiveness study was not designed to examine 
the nature, extent, or effectiveness of testing programs or 
screening procedures in current use. The study was designed to 
avoid labelling of children, self-fulfilling expectations, and 
intrusion into the children's educational programs. The study 
was not designed to examine the effectiveness of testing programs 
and screening procedures in current use. 

The children's scores on the multiple indicators of the 
kindergarten testing program were used as the pre-test portion cf 
the design. The teacher judgment instrument and the three 
commercially published screening tests were described m the 
previous chapter. The post-test portion of the design was 
composed of two measures of reading achievement: teacher ludgmenr 
and a nationally used standardized reading achievement test. The 
teacher judgment instrument relied on the teacher's daily 
observations of the child's performance in the classroom over a 
sustained period of instruction. The standardized reading 
achievement test instrument relied on the child's performance 
during a particular testing session. 

For purposes of the effectiveness study, several operational 
definitions were adopted. "Potential reading problems" was 
defined as scoring below the cutoff point on both of the multiple 
indicators used in the pilot project's kindergarten testing 
program and administered in spring 1986. "Reading achievement 
deficit" was defined as scoring below the cutoff point on one or 
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both measures of reading achievement administered in spring, 
1988. "Effectiveness" was defined as meeting the criterion of 
having correctly identified at least 75% of project children in 
kindergarten as having potential reading problems. The criterion 
of 75% was adopted on the basis that 50% is a chance level; 75% 
is 50% greater than chance. In other words, for the pilot 
project's kindergarten testing program to meet the criterion of 
effectiveness adopted for the study, at least 75% of the children 
identified by both of the multiple indicators in kindergarten as 
having potential reading problems had to demonstrate actual 
reading achievement deficits at the end of the two-year period. 

Instrumentation . The two pre-test indicators, kindergarten 
teacher judgment and commercially published screening tests, \ad 
been administered in the kindergarten testing program. A 
project-designed teacher evaluation form ("Year Three Teacher 
Evaluation Form," so named for the third year of the pilot 
project [California State Department of Education, 1938]) and 
Form U, Level D of the Comprehen sive Test of Basic Skills ( CTgS ) 
were selected as the post-test measures of raading achievement. 
The teacher evaluation form measured the study participants' 
reading achievement level compared to the average pupil in second 
grade. The form rating was scaled from 1 to 5. A rating of 1 
indicated that the child's reading achievement level was 
extremely poor compared to the average child in second grade. 
The rating of 2 meant that the child was below average, while a 3 
indicated that the child was average. A rating of 4 indicated 
that the chilu was above average, and a rating of 5 indicated the 
child was excellent. The CTBS reading achievement test yields a 
percentile rank from 1 to 99. 

Cutoff Points, cutoff points were established for the pre— cest 
indicators of potential reading problems and the post-test 
measures of reading achievement deficit. Cutoff points were 
adopted as recommended by the technical subcommittee of the group 
of field experts (see Appendix B) . Because of the variety of 
scores yielded by the four selected screening measures, different 
-types of cutoff points had been established for the commercially 
published screening tests and the kindergarten teacher evaluation 
form. For the pre-test indicator of the commercial screen, two 
cutoff points had been selected, the nigheec score within che 
bottom 15% of the scores made by project children in spring 1986 
on each commercial screen as one cutoff point for those screens, 
as well as the highest score within the bottom 25% of the scores. 
For the pre-test indicator of kindergarten teacher judgment, the 
lowest rating of 1 had been previously selected. 

For the post-test measure, CTBS reading achievement test, the 
national rank at the 20th percentile was established as the cut- 
off point. For the "Year Three Teacher Evaluation Form," the 
lowest rating of 1 was chosen as the cutoff point. 
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D ata Collection. Ratings and score* from the pre-test 
instruments were gleaned from the files of the pilot project's 
Kindergarten testing program. From spring, 1986 through spring, 
1988. iiitensive efforts were carried on by the schools in which 
pilot project children were enrolled, so that the pilot project 
could keep in touch with the schools to which the children 
transferred. These child tracing efforts were conducted hy the 
pilot project in cooperation with the eight school districts in 
which the testing program had been implemented during spring, 
1986. In addition, the tracing efforts were conducted with a 
number of other schools to which the children who had 
participated in the kindergarten testing program transferred 
during the ensuing two years, through spring, 1988. The purpose 
of these tracing efforts was to assure that as many children as 
possible would be available to participate in the spring, 1988 
post-test data collection. 

In the spring, 1988, post-test data collection support materials 
were created for local use. These included: a comprehensive set 
of instructions and log sheets for use by the coordinators in the 
eight original districts; a comprehensive set of instructions and 
log sheets for use by principals in schools outside the eight 
original school districts; rhe project-designed "Year Three 
Teacher Evaluation Form"? and supplementary testing instructions 
to be used with the CTBS , Form U, Level D, Grade 2 reading 
achievement test. By the end of spring, 1988, schools had 
administered both of the post-test reading achievement :neasures 
to 923 participating children. 

Preliminary Data Analysis. Before the pre-test and post-test 
data from the 923 remaining children were analyzed, the pre-test 
data were analyzed to determine the effects of attrition 
(incomplete data and untraceable children) on the population of 
study participants. Descriptive statistics were calculated on 
the pre-test instruments (the multiple indicators of potential 
reading problems) of the 1,180 original study participants and 
then again on the ore-test instruments for the group of 92 3 
remaining study participants at the end of the two-year period. 

Effects of Attrition. At the end of the two and one-half years 
of student tracing, study students had been enrolled in over 4 00 
schools in 185 school districts in California and in other 
states. A total of 95 study participants were untraceable during 
the two years, yielding an attrition rate of 8.1% over the two 
year period. For analysis, a complete data case contained four 
elements; 1) a commercial screening score? 2) a kindergarten 
teacher rating; 3) a CTBS , Form U, Level D Total Reading 
Achievement percentile rank; and 4) a year three teacher rating. 
Complete data on study participants were available for 923 study 
participants, or 78-2% of the original group of study cnildren 
who had been screened by the kindergarten testing program at the 
end of kindergarten in the spring of 1986. 
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Selected deacriptive statistics (number of participants tested; 
and lowest score r highest score, nean score, ard standard 
deviation of the set of pre-test instnments) for those 1,180 
study participants screened by both indicators in the spring, 
1986 and for the 923 study participants for whoa there were 
complete data in the spring, 1988 can be found in Appendix D. 
For the FLORIDA it will be noted that the lowest and highest 
scores exceed the range given in the description of the 
instrument in the previous chapter. The outlying scores appear 
to be artifacts of the calculation process required by the test 
publisher. 

The effects of the attrition rate (8.1%) and the incomplete data 
cases (13.7%) can be seen when comparing the two tables in 
Appendix D. Each commercial screen kept relatively the same 
proportion of study participants over the two-year period. The 
mean score for each commercial screen did not change appreciably. 
For example, the spring, 1986 mean score on the kindergarten 
teacher evaluation instriiment was 3.26 for the 1,180 original 
participants and was 3.36 for the 923 participants remaining in 
the pilot study in spripg, 1988. The effects of attrition 
appeared to be negligible. 

Data Analysis. To determine the effectiveness of the pilot 
project's kindergarten testing program, pre-test and post-test 
data were analyzed for each child. Cross-tabulations of the pre- 
test and post-test data were done for each of the 923 children 
for whom the study had complete data. In summary, for the 
multiple indicators (kindergarten teacher judgment and commercial 
screens) the following cross-tabulations were done: kindergarten 
teacher rating and commercial screen score versus year three 
teacher rating and CTBS rank; kindergarten teacher rating and 
commercial screen score versus year three teacher rating; and 
kindergarten teacher rating and commercial screen score versus 
CTSg rank. Separate cross-tabulations were performed using the 
two cutoff points for the commercial screen scores as described 
earlier. 

In addition, cross-tabulations were performed for each of the 
pre-test single indicators for the 923 children. In summary, for 
the pre-test and post-test data, the following cross-tabulations 
were done: kindergarten teacher rating versus year three teacher 
rating; kindergarten teacher rating versus CTBS rank; commercial 
screen score using cutoff of the lowest 15% of the scores versus 
year three teacher rating; commercial screen score using cutoff 
of the lowest 15% of the scores versus CTBS rank; and commercial 
screen score using cutoff of the lowest 25% of the scores versus 
year three teacher rating; commercial screen score using cutoff 
of the lowest 25% of the scores versus gTPg rank. 
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Finding^ 



This section contains separate subsections with the findings on 
the multiple indicators and on the single indicators used in the 
pilot project's kindergarten testing program. 

Findings on Multiple Indic ators of the Pilot Project's 
Kindergarten Testing Program. As displayed in Taole 1, the pilot 
project's kindergarten testing program was partially effective in 
meeting the 75% criterion. 

The Kindergarten Testing Program using both of the indicators was 
not effective when both post-test measures of reading 
achievement, teacher rating and standardized test, were used as 
evidence of reading achievement deficits. 

The Kindergarten Testing Program was effective when CTBS 
standardized test was used alone as evidence of reading 
achievement deficits. 

The Kindergarten Testing Program was not effective when teacher 
rating at the end of the two year period was used alone or in 
combination with the CTBS . 

Raising the cutoff on the commercial screen scores from the 
lowest 15% of the scores to the lowest 25% of the scores did not 
raise the percent of correct identifications at the end of the 
two year period. In other words, widening the net at the end of 
kindergarten yielded a lower percent of correct identifications. 



Table 1 



Effoctivaneas of the Kindergarten Testing Program 
in Correctly Identifying Children 
as Having Potential Reading Problems 
(Multiple Pre-Test Indicators) 



Pre -test 
Indicators 



Kindergarten 
Teacher Rating 
& Screens (15%) 

Kindergarten 
Teacher Rating 
& Screens (15%) 

Kindergarten 
Teacher Rating 
& screens (15%) 
Kindergarten 
Teacher Rating 
& Screens (25%) 

Kindergarten 
Teacher Rating 
& screens (25%) 

Kindergarten 
Teacher Rating 
& Screens (25%) 



Number of 

Children 

Identified 

Spring '86 



40 



40 



40 



53 



53 



53 



Post-Test 
Measures 



Year Three Teacher 
Rating and 
CTBS Rank 



Year Three 
Teacher Rating 
Only 



CTES Rank Only 



Number of 
Children 
with Actual 
Deficits 
Spring • 88 



18 



18 



37 



Year Three Teacher 
Rating and 
CTBS Rank 



Year Three 
Teacher Rating 
Only 



CTBS Rank Only 



23 



23 



44 



Percent 
of 
Correct 
Identification 



45.0% 



45.0% 



92.5% 



43.4% 



43.4% 



83. 0% 
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Findings m Sjnale Indicators of tha Pilot Proj ect's Ki ndergarten 
Testing Program. As displayed in Table 2, no single indicator 
used in tii*» Kindergarten Testing Program met the 75% criterion of 
correct idtntif ications. 

All single indicators yielded a higher percent of correct 
identifications when the CTBS percentile rank was used alone as 
evidence of reading achievement deficits. 

Kindergarten teacher judgment approached the 7 5% criterion when 
CTBS percentile rank was used alone. 

The commercial screens using either cutoff point (lowest 15% of 
the scores or lowest 25% of the scores) did not come close ro the 
75% criterion when using either of the post-test measures. 
Kindergarten teachers identified fewer children as having 
potential reading problems than did commercial screens. Overall, 
kindergarten teacher judgment was a better indicator of potential 
reading problems, as compared to the commercial screens, no 
matter which cutoff point was used (lowest. 15% of the scores or 
lowest 25% of the scores) . 

Raising the cutoff on the commercial screen score from the lowest 
15% of the scores to the lowest 25% of the scores did not raise 
the percent of correct identifications at the end of the two-year 
period. In other words, widening the ner at the end of 
kindergarten yielded a lower percent of correct identifications. 
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Table 2 



Effectiveness of the Kindergarten Testing Program 
in Correctly Identifying Children 
as Having Potential Reading Problems 
(Single Pre-Test Indicators) 



Pre-test 

T T\(\ 4 t" n T* ^ 

JitilV* J»WC4 wVJX 9 


Number 
Children 
Tdentif lad 

spring *86 


Post-Test 
Me&sures 

A A^S V Am Vw m9 


Number of 
Children 

Deficits 
Spring '88 


Percent 

of 
Correct 
Identification 


Kindergarten 
Teacher Rating 


82 


Year Three 
Teacher Rating 


30 


36.6% 


Kindergarten 
Teacher Rating 


82 


PT?S Rank 


60 


73.2% 


screens (15%) 


120 


Year Three 
Teacher Rating 


38 


31.7% 


Screens (15%) 


120 


CTBS Rank 


81 


67 .5% 


Screens (25%) 


220 


Year Three 
Teacher Rating 


52 


23 . 6% 


Screens (25%) 


220 


CTBS Rank 


116 


52.7% 



Discussion 

The findings on the accuracy of identification by the screening 
. measures show that the concerns expressed by parents and 
education professionals about misidentif ication^ as described in 
Chapter 1, are well-founded. As can be found in Appendix the 
pilot project's kindergarten testing program made both correct 
identifications and incorrect identifications. The incorrect 
identifications made by the testing program were of two kinds: 
some children were incorrectly identified as having potential 
reading problems, and actually had no indicated difficulties two 
years later. Other children were incorrectly identified as not 
having potential reading problems, and actually had such 
difficulties two years later. This may be due to the 
developmental diversity of children at the kindergari:en age span; 
with their differing rates of growth, development, and 
maturation; and to the resulting unreliability of children's 
performance at that age level. 
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Summary 



Five major findings emerged from the analyses of the data. 
First, a small number of kindergarten children was identified by 
both of the multiple indicators of the pilot project's 
kindergarten testing program. The small number is reflective of 
the relatively small number of children identified by their 
kindergarten teachers as having potential reading problems, as 
compared with the number of childrim identified by the commercial 
screening instruments. 

second, the pilot project's kindergarten testing program was 
partially effective in having correctly identified children in 
kindergarten as having potential reading problems. The multiple 
indicators of the kindergarten testing program (kindergarten 
teacher judgment and commercial screening instrument) met the 
criterion of 75% correct identifications only when the children's 
scores on a standardized test of reading achievement was used as 
the post-test measure. 

The kindergarten testing program did not meet the criterion of 
75% correct identifications when teacher judgment of the 
children's reading achievement was used as the post-test measure. 

Third, no single indicator of the kindergarten testing program 
met the criterion of 75% correct identifications, regardless of 
the post-test measure used as evidence of actual reading 
achievement deficit, teacher's judgment, or children's scores on 
the standardized test of reading achievement. 



Fourth, of the single indicators of potential reading problems, 
the kindergarten teacher judgment instrument came closest to 
meeting the criterion of 75% correct identifications. The 
teacher judgment instrument approached the criterion only wnen 
the standardized test of reading achievement was used as the 
post-test measure. 

Fifth, raising the cutoff point on the commercial screening 
instrument indicator of potential reading problems lowered the 
percentage of correct identifications. In other words, widening 
the net in kindergarten resulted in a lower percentage of correct 
identifications by the pilot project's multiple indicator 
kindergarten testing program. 

The next chapter presents the Department's conclusion on the 
feasibility of implementing the kindergarten testing program 
statewide at the end of the pilot project. 
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Chapter IV. 
Feasibility of Implementing 
the Kindergarten Testing Program Statewide 



tm,^ r-y^^ry^t^y nrasents the Department's conclusions on the 
^L?bili?v of implLen^ing the Kindergarten Testing Program , 
o^^^iwide ^heolpartoent adopted three criteria for determining 

«^ the end of the pilot project*, effectiveness, statewiae 
StiU^Y? and cost. The Department's conclusions on feasibility 
^^iii on IV to the kindergarten testing program developed and 
SSiLeStL by tL p^^ Ire not intended to apply to 

t«i!nrprogrLrand screening procedures in current use. The 
^oartSeStTconclusion on each of the criteria is presented in 
^pa^^e sections, followed by a 8^mmary and recommendation. 

vi^^'^ni■i^r^n&ss As Stated in the previous chapter, the pilot 
oro iect'^k^nd ergar^en testing program was partially effective in 
correctly identifying children with potential reading problems at 
an e!rS period. The multiple indicators of the kindergarte. 
?es?ing program (kindergarten teacher judgment and commercial 
screening instrument) met the criterion of 75% correct 
iSn?if icationronly when the children's scores on a standardized 
tesf of reading achievement were used as the post-test measure. 

The multiple indicators of the kindergarten testing program did 
l^t m««t ?he criterion of 75% correct identifications when 
?eLSefjS^ent of ?L Children's reading achievement was used 
as the post-test measure. 

No sinale indicator of the kindergarten testing program met the 
c?i?e?ion of 75% correct identifications, regardless of the post- 
?est measure used as evidence of actual reading achievement 
deScut the teacher's judgment or the children's scores on the 
standardized test of reading achievement. 

Of the single indicators of potential reading Pf°^^«J*;^'^|;^^^ . „ 
kindergarten teacher judgment instrument came clo»«st to meeting 
the criterion of 75% correct identifications. The teacher 
judgment Instrument approached the criterion only f « . . 

standardized test of reading achievement was used as the post- 
test measure. 

statewide utilitv. The commercially published screening tests 
werl n it designed for use with the rapid' y changing demographic 
of children enrolled in California public schools. In 
?98rSlilorSia hTovSr one million language minority students; 
that is onrout of four students. The future population 
prSjec?ioS2 show an increasingly diverse linguistic community 
throughout California. 



Table 3 



Estimated Costs of Implementing 
the Pilot Project's Kindergarten Testing Program Statewide 



Field Activity 



Per-pupil 
cost in 
pilot project 



Estimated 
cost for 
statewide 
implementation 



Orientation to testing 
and time for teachers 
to rate their pupils 



Purchase of commercial 
testing materials 



$ 0.81 



$ 1. 67 



317,610-72 



V 



654 ,827. 04 



Training test 
examiners 



$ 1.71 



$ 670,511.52 



Administration and 
scoring of tests 



$ 18.06 



$ 7,081,542.72 



Travel for examiners 
and monitor 



$ 5.39 



$ ,113,483-68 



Duplication of 
teacher judgment 

form $ 0,01 $ 3,921-12 



Total Estimated Costs 



$ 27.65 



$10 ,341,896.30 



Cost^ The costs for statewide implementation of the kindergarten 
testing program are estimated at $10,841,897. This estimate is 
based on the per-pupil costs of the field activities of the pilot 
project in implementing the testing program for 1,187 pupils in 
spring, 1986. In Table 3 those per-pupil costs have been 
multiplied by 392,112, the number of pupils in kindergarten m 
1987, the most recent data available. 

' I addition to the costs incurred in the pilot project, there 
would be costs at local and state levels for practical 
implr.mentation of the kindergarten te£* ing program. At the local 
level, there would be unknown costs tc school districts for staff 
time to develop local cutoff points, obtain parent permission to 
test children, select examiners, interpret children's scores, 
writing individual reports on each child's results, and informing 
each child's parents by mail, personal conference, or telephone 
of the child's performance. Administrative costs would be 
incurred at the district level for establishment of district 
guidelines, assistance to schools in the implementation, and 
reporting to school boards and the Department on the costs and 
the results. 

Administrative costs of an unknown amount would be incurred at 
the state level. The Department would need to establish 
statewide guidelines or regulation, or both; provide assistance 
to districts in start-up; monitor districts and schools for 
compliance; calculate the amounts of any appropriations requests 
necessary to implement a new state mandate; calculate the amount 
due to each district; apportion state monies; review district 
results; and audit district fiscal reports. 

summarv and Recorftmendation. In light of the limited 
effectiveness of the pilot project's kindergarten testing 
program, the growing numbers of limited-English proficient pupils 
who would be excluded from this testing program, and the 
projected per-pupil costs of statewide implementation of the 
testing program, the Department concludes that statewide 
implementation is not feasible. 

In view of the relative greater effectiveness of the kindergarten 
teacher judgement indicator of potential reading problems and the 
relative lower cost for administering the kindergarten teacher 
judgement procedure, the Department recommends further 
developmental work to increase the effectiveness of kindergarten 
teacher judgment as one means of correctly identifying children 
with potential reading problems. Such developmental work should 
take in account the wide diversity of children's developmental 
patterns and the linguistic and cultural diversity of children in 
California's public schools. 
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Appendix A 

Chapter 1376, Statutes 1985 (Assembly Bill 972) 



Article 13 i^ariy Uiagnosui of Ix^iiriiing Disabilities 



48680. The State Department of Education shall develop a 
tcfling program to bi; utilized at the kindergarten grade level to 
detormine which pupils have a potential for developing learning 
dif^ibty proWenis, Tlie testing procedure shall include an overall 
•Greening test for leak ning disabilities and testing for dyslexia. To the 
extent feasible, the department shall use existing tests and screening 
inftnunents in developing the early diagnosis of the learning 
dittbilities testing program. In developing the program, the 
department shall consult with experts in the areas of learning and 
reiding difficulties, including, but not limited to, neurologists, 
p^rciialog;ists, persons working in these areas in pt>stsecondary 
educational institutions, teachers, school nurses, education 
conmltants, school i^sychologists, and other persons with appropriate 
knowledge and expt^rience in the detection aiid treatment of 
learning |wrdblenis and reading tlifficulties in early grades. 

40S81. The State Department of Education shall d^evelop and 
in^4ement a pilot project to determine the effectiveness and 
fcaiibdity of implementing tha early diagnosis of learning disabilities 
t«iting program developed pursuant to Section 49580. The pilot 
prc^ shall administer the early diagnosis testing program to 
kindergarten aged pupils in order to identity pupils with llu* 
potential to develop learning disability probh'ins Pupils who ar<' 
identified as having |H)tenlial learning disability prohh-iiis shall ht- 
referred to existing piogiams and services which arr availabK- to 
provide assistant <■ 

49S82. On or iH toit- January I, liftHh the HUilr Dt partnu nt ot 
Education shall presciil>e guideUnes for the early <liagnosis of ilw. 
learning <hsabilitit s testing program and pilot project. The giiitleliiu s 
shall include, hut need not U* liituted to, all of the following. 

(a) A definition of "pupils with the i)<)tential to de velop learning 
diMbility problenis. " as used in this article 

(b) The meth«Kls and criteria for selet tiiig one oi iiiocc sitt s loi 
die establishment ol llu- pih>t pioiect 

(c) Theniiinb.M oi >il< s to ho sel<-t iU.M jMiipw ,« ol r^lahliNlmiK, 
the pilot project 

(J) Ciiteria for judging llu n suits and t rterliv i lu ss *>! thu t ail> 
diicgnosis testing program, as well as crit^'ria tor deteiniinin}? the 
feasibility for iinjjUnienting tlit- piograin at ihr roiu hisiorj of tlx- 
pilot project. 



49S83. On or before January 1, 1989, the State Department of 
Education shall submit a report to both etlucational policy 
committees of the I legislature regarding the results of the early 
diagnosis of the learning disabihties pilot project. The report shall 
include, but need not be limited to, an assessment of the degree to 
which the early diagnosis of learning disabilities in pupils 
participating in the pilot program resulted in a reduction of the need 
for later remedial education for thos*' pupils, and the projected cost 
effectiveness of this early diagnosis. 

SEC. 2. The sum of one hundreil fifty thousand dollars ($150,000) 
is hereby appropriated trom the General Fund to the State 
Department of Education for allocation in accordance with the 
following schedule: 

(a) Vot purposes of developing a testing program for 
the early diagnosis of learning disabilities 
pursuant to Article 13 (commencing with St>ction 
49580) of Chapter 9 of Part 27 of the Education 

Code in the 1985-^ fiscal y«'ar $ 5().(K)0 

(b) Fur purposes of validating the testing program 
destrilHHl in sulnlivision (a) in th<* P)«(>-87 and 
h>87-8« fiscal years, with filiy ihonsaiid dollars 
i^riO.im) to he ull<K att <l to the d« pai timut in 

cai h fiscal year $1(HM«M> 

Six: 3 This act h an urgency statute necessary for the 
immediate prest»rvation of ihe public |Kace, h<'alth, or safety within 
the meaning of Article IV of the Constitution and shall go into 
nniiiediate effect. The facts constituting the necessity are. 

In order to recjuire the State Department of F,<lueation to develop 
a t<*slfiig progranj for the early diagnosis of l» ;ti niiig disabilities at the 
earliest jxtssihle time, it is lU'crssary thai this act take effect 
inMncdi.itcly 
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Appendix B 



List of Field Experts 



Listed in the paragraph below in alphabetical order are the field 
experts with their primary positions and institutions, as 
reported in the summer of 1988: 

Beverly Bigler, M.S., Ed.D., School Nurse/Pediatric Nurse 
Practitioner-certified, Alhambra School District? Jeffrey L. 
Black, M.D., Assistant Clinical Professor, Department of 
pediatrics. University of California, San Diego; Nicholas A. 
Bond Jr., Ph.D., Professor, Department of Psychology, California 
State University, Sacramento; Elmer c. Cameron, M.A. , Associate 
Superintendent, Supplementary Education Services, Escondido Union 
school District; Christopher H. Chase, Ph.D., Research 
Associate, Language Research Center, Children's Hospital, San 
Diego; Boyd A. Johnson, M.A. , Director, Diagnostic Educational 
Success Center, Redlands; Barbara K. Keogh, Ph.D., Professor of 
Educational Psychology, Graduate School of Education, University 
of California at Los Angoles? Delmont C. Morrison, Ph.D., 
Clinical Professor, Department of Psychiatry, Langley Porter 
Institute, University of California, San Francisco; Jean C. 
Ramage, Ph.D., Professor, Department of Counselor Education, Co- 
director of School Psychology Program, San Diego state 
University; Nancy L. Royal, Ed.D., Director, Prentice Day 
school, Costa Mesa; Paul Satz, Ph.D., Professor of Medical 
psychology and chief, Neu" ^psychology Program, The 
Neuropsychiatric Institute and Hospital, University of 
California, Los Angeles; Mary L. Scholl, M.D., Associate 
Clinical Professor of Neuroscience, University of California, San 
Diego; Paul Schultz, M.D. , Co-Director, Neurology, Children's 
Hospital, San Diego; Lillian Stillwell, M.A., Director, Pupil 
Personnel Services, Retired, Auburn Elementary School District; 
Paula Tallal, Ph.D. Co-director, Center for Molecular and 
Behavioral Neuroscience, Rutgers University; Barbara F. Thomas, 
M A. Director, Special Education Local Plan Area, Fresno County 
Office of Education; Claude A. Valenti, O.D., FCOVD, Director, 
Optometric Vision Development Center, La Jolla; Mary Zieglar, 
B.S., Project Coordinator, Language Research Center, Children s 
Hospital Research Center, San Diego. 

The following members of the group of field experts comprised the 
technical subcommittee: ^ ^ , . ^ 

Christopher H Chase, Ph.D.; Barbara K. Keogh, Ph.D.; Delmont C. 
Morrison, Ph.D.; Paul Satz, Ph.D.; Mary Ziegler, B.S. 
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List of Selected References 



California State Department of Education, "Year Three Teacher 
Evaluation Form," 1988, unpublished. 

Chapter 1376, statutes of 1985 (Assenbly Bill 972) 

Enrollment Data; California Elementary a^nd ??gQPaarY V^lic 
Schools 1987-88. The California Basic Educational Data System. 
Sacramento; California State Department of Education, 1988. 

Glass, Gene V. "Standards and Criteria," Journal of Educational 
Measurement , Vol. 15, No. 4 (Winter 1978): 237-61. 

Goodwin, William L. , and Laura A. Driscoll. ?iahdt?ooK for 
Measurement and Evaluation i n Earlv Childhood Education. San 
Francisco, California: Jossey-Bass Publishers, 1980. 

Here Thev Come: Ready or Notl Report of the School Readiness Task 
Force. Prepared by the Piiblic and Governmental Policy Branch. 
Sacramento: California State Department of Education, 1988. 

Kelly, Thomas F. "Reflections on High Expectations: There Is No 
such Thing as an Overachiever . " Article transmitted through 1988 
personal correspondence, Dix Hills, New York: Suffolk III BOCES., 
n.d. 

Keogh, B. K. , and M. W. Sbordone. "Early Identification of High 
Risk and High Potential Kindergarten Children." A technical 
report prepared under contract between the California State 
Department of Education and the University of California, Los 
Angeles. December, 1974, p. 63. 

Learning Disabilities: A Repor t To The U.S. Congress. Prepared by 
the Interagency Committes on Learning Disabilities. Washington 
D.C: The Department of Health and Human Services, 1987. 

Lichtenstein, Robert, and Harry Ireton. Preschool Screening: 
identifving Young C hildren with Developmental and SdUPational 
Problems . Orlando, Florida: Gene and Stratton, Inc., 1984. 

Martin, Anne. "Screening, Early Intervention, and Remediation: 
Obscuring Children's Potential", Harvard E ducational Review, Vol. 
58, No. 4, Nov. 1988, pp. 488-501. 

Meehl, Paul E. , and Albert Rosen. "Antecedent Probability and 
the Efficiency of Psychometric Signs, Patterns, or Cutting 
Scores," Psvchologjcal Bulletin , Vol. 52, No. 3 (1955): 194-216. 
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Meehl, Paul E. gl j n i vs . St^X i j c ^l ^^y^t^fi l" ^Unlversitf of^ 
^pnlvais and R« ^Yi^^ ^>^*> Evidence. Minneapolis. University ot 

Minnesota Prcbu, 1956. 

Satz, Paul, and Jack Fletcher. Manual fgr tfte Florida 
^?pL.^;^,;n scr^^n^nc Batterv. Odessa, Florida: Psychological 
Assessment Resources, Inc., 1982. 

^cTf^^nXm 9t .?chQPl ;4}f^^^g^"^gfl^c.iB?^ture°S^ 

Expenditure Review Program Audit. Albany: The Legislature or i^ne 

State of New York, 1985. 

silver Archie, and Rosa Hagin. search: A Scanning Instruffgnt toy 
^^i^?S;n^ flecti on Of potent ial J,aam i nq Pi^^friUtY . 2d ed.,enl. 
New York: Walker Publishing Company, 1981. 

slingeriand, Beth H. ?re Reading ^^s. 
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Educators Publishing Service, Inc., 1977. 



26 



Appendix D 
Selected Descriptive Statistics 



Year One (1986) 
Selected Statistics 
of Study Participants in Kindergarten 
by Pre-Test Screening Instrument 





Screening Instrument 






SEARCH 


SLINGERIAND 




Number of 
screened 


395 


405 


380 


1180 


Lowest Scone 


-3.77 


2.00 


0.00 


1.00 


Hi^est Sccre 


3.89 


! 

9.00 1 130.00 


5.00 


Mean Score 


0.53 


7,65 


96.71 


3.26 


Standard 
Deviation 


1.06 




18.75 





Selected Statistics 
of Study Participants 
Peraaining in Year Ihree (1988") 
by Pre-Test Screening Instrjr.ant 







Screening 








SEARCH ; 


! 

SIZIJGEPIAND t 




Nmnber of 

childnen 

screened 


303 


224 


296 


923 


! 

Lowest Score 


^1 77 


i 

2,00 


i 




Highest Score i 


2.39 


9,00 j 


130.00 




I'tean Score 


^ * 




98.06 




Standard 
Deviation 




t — ^ ■ 


_ / . 06 





Appendix D ( cont . ) 



Ef fectiveiess of the KindBrgarten Testing Pnxfram 
in Identifying Qiildren as Hatving 
Potential Reading Frcblens 
(Multiple Pre-test indicators [Screens 15%] 
and OoBbined Post-'test Maasures) 



NUober of Qiildren 
Identified as Having 
Potential Readincr Prt±jleros 


Spring 1988 
Reading Adiievenent status 
of CJiildren 


Treacher JtxigeBent & 




Screens at 15% 




18 ciiildren 


40 ciiildren 


Actual Reading Aciuevonent Deficits 




22 children 




No Actual Reading Aciiievenent Deficits 



Effectiveness of tne Kindergarten Testing Program 
in Identifying Oiildren as Not Having 
Potential Reading Problems 
(Multiple Pre-test Indicators [Screens 15%] 
and Ccntoined Post-test Measures) 



Spring 1986 
Nuntoer of Oiildren 
Identified as Not Having 
Potential P^lm ?nnblgms 


Spring 1988 
Reading Achievenent Status 
of Children 


Teacher Judgement & 




Screens at 15% 






732 diildren 


883 children 


No Actual Reading Achievenjent Deficits 




151 children 




Actual Reading Achievement Deficits 
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Appendix D ( cont . ) 



Ef f ectivBness of thft Kindergarten Testing Prtsgram 
in Idtttti^dng Children as Having 
Potential Reading Prcblens 
(Multiple Pr»-test Indicators [Screens 25%] 
and Oartoined Post-test Measures) 



Spring 1986 
Nurtwr of Children 
Identified as Having 
Potentiza Raadina Prcfclesas 


Spring 1986 
Reading Achieveraent Status 
of Children 


Teacher Judgement & 




Screens at 25% 






23 children 


53 ciuldren 


Actual Reading Achievement Deficits 




30 children 




No Actual Reading Achievement Deficirs 



Effectiveness of the Kindergarten Testing Program 
in Identifying Children as Not Having 
potential Reading Problems 
(Multiple Pre-test Indicators [Screens 25%] 
and Corhbined Post-test >feasures) 



Spring 1986 
Number of Children 
Identified as Not Having 
Potential Readina Problems 


Spring 1988 
Reading Achievement Status 
of Children 


Teacher Judgement & 




Screens at 25% 






720 children 


870 children 


No Actual Reading Achievement Deficits 




150 children 




Actual Reading Achievement Deficits 



Appendix D (cont.) 



Effectiveness of the Kindergarten Itesting Program 
in Identifying Oiildwn as Haying 
potential Reading PrdbI«0B8 
(Miatiple Pre-test indicators (Screens 15% ] 
ani CTB5 Post-^test Measure) 



Spring 1986 
miuiDer ox uixxorcai 
Identified as Having 
Potential Beading Prcblews 


Spring 1988 
Seadina Adhievwnent Status 
of Qiildren 


I^aaciier Judgement & 




Screens at 15% 






37 children 


40 children 


Actusd. Reading Achieveraent Deficits 




3 children 




No Actual Reading Achievement Deficits 



Effectiveness of the Kindergarten Testing Program 
in Identifying Children as Not Having 
Potential Reading Problems 
(Multiple Pre-test Indicators [Screens 15%] 
and CTBS Post-test Measure) 



Spring 1986 | Spring 1988 

Number of Children | Reading Achievement Status 

Identified as Not Having | of Children 

Potential p^wjjTvr P rdblems I 

Teacher Judgement & I 

Screens at 15% | 

' 718 children 



883 ciuldren 


No Actusd. Reading Achievement Deficits 




165 children 




Actual Refilling Achievement Deficits 
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Appendix D (cont.) 



Effectiveness of the Kindercfarten Testing Program 
in Identifying Qiildren as Having 
Potential Beading Pmhiems 
(Multiple Pre-test Indicators [Screens 25%] 
and CTBS Post-tast Measure) 



Spring 1986 
Nvacber of Qiildren 
Identified as Having 
Potential Peadina Problens 


Spring 1988 
Reading Achievement Status 
of Children 


Teaciier Judgement & 




Screens at 25% 






44 children 


53 ciiildren 


Actual Beading Achievement Deficits 




9 children 



No Actual Reading Achievenjent Deficits 



Effectiveness of the Kindergarten Testing Program 
in Identif^'ing Oiildren as Not Having 
Potenticil Heading Prcblems 
(Multiple Pre-resr Indicators [Screens 25%] 
and CTBS Post-test Measure;) 

Spring 1986 '■ Sprir>g 1988 

Number of Children Reading Achievement Starus 

Identified zs Not Having , of Children 

Potential Readinc Problems 

Teacher Judgement & 
Screens ar 25% 

712 children 

870 children * No ActusLL Reading Achievemenr Deficirs 

; 158 children 

Actual Reading Achievement Deficits 



Appendix D (cont*) 

Search Screen (25%) 
Versus 

CTBS Reading Achievement (20th %-ile) 

n-324 



Screen 


CTBS < 21st %-ile 


CTBS > 20th %-ile 


Indicator 


Indicator 


Indicator 


Potential 






Peftdir^q Probleins 


49 


42 


No Potential 






Readina Problems 


20 


213 


Total 


69 


255 



Slingerland Screen (15%) 
Versus 

CTBS Reading Achievement (2 0th %-ile) 

n»296 



Screen 


CTBS < 21st %-ile 


C^BS > 20th %-ile 


Indicator 


Indicator 


Indicator 


Potential 






Readina Problem? 




10 


No Potential 






Readina Problems 


29 


231 


Total 


'5 


241 



Slingerland Screen (25%) 
Versus 

CTBS Reading Achievement (20th %-ile) 



Screen 


CTBS < 21st %-ile 


CTBS > 20th %-ile 


Indicator 


Indicator 


Indicator 


Potential 






Readina Problems 


33 


32 


No Potential 






Readina Problems 


22 


209 


Total 


55 


241 



Appendix D (cont.) 

Florida Screen (15%) 
Versus 

CTBS Reading Achievement (20th %-ile) 

n-303 



Screen 
Indicator 


grS? < 21st %-ile 
Indicator 


(pTBS > 20th %-ile 
Indicator 


Potential 
Readina Problems 


23 


14 


No Potential 
Readina Problems 


55 


211 


Total 


78 


225 


Florida Screen (25%) 
Versus 

CTBS Reading Achievement (2 0th %-ile) 

n=303 


Screen 
Indicator 


CTBS < 21st %-ile 
Indicator 


CTBS > 20th %-ile 
Indicator 


Potential 
Readina Problems 


34 


30 


No Potential 
Readina Problems 


44 


195 


Total 


78 


225 


Search Screen (15%) 
Versus 

CTBS Reading Achievement (20th %-ile) 

n-324 


Screen 
Indicator 


CTBS < 21st %-ile 
Indicator 


CTBS > 20th %-ile 
Indicator 


Potential 
Readina Problems 


32 




No Potential 
Readina Problems 


37 


240 


Total 


69 


255 
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